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Abstract 



The Tevatron proton-antiproton collider at Fermilab with its centre of 
mass energy of 1.96 TeV allows for pair production of top quarks and the 
study of top quark decay properties. This report reflects the current sta- 
tus of measurements of the W boson helicity in top quark decays and the 
ratio of top quark branching fractions as well as searches for neutral cur- 
rent top quark decays and pair production of fourth generation t' quarks, 
performed by the DO Collaboration utilising datasets of up to 5.4 fb" 1 . 



1 Introduction 



With a mass of m t — 173.2 ± 0.9 GeV [T], the top quark is the most massive 
fundamental particle known to date. Top quarks are mainly produced in pairs 
via the strong interaction, with a cross section of a t i — 7.3lg'J pb [2] in pp 
collisions at y/s = 1.96 TeV. In the framework of the Standard Model (SM), top 
quarks decay almost exclusively into W bosons and b quarks. Consequently, top 
quark pair events contain a b and a b quark from the it decay, and the remaining 
final state to be observed depends on the decay mode of the two W bosons. 

The analyses presented here focus on two signatures: (i) events where both 
W bosons decay leptonically, resulting in a final state containing two isolated 
high-py leptons, missing transverse energy $t corresponding to the two neu- 
trinos and two (&-) jets (dilepton events) and (ii) events where one W boson 
decays leptonically, the other one hadronically, resulting in one isolated high-p^ 
lepton, TfiT and four jets (lepton + jets events). As leptons only electrons and 
muons are considered in these analysis channels, including those from leptonic 
t decays. 
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2 Searches for New Physics in Top Quark Pair 
Decays 

Measurement of the W Boson Helicity 

Top quark decays in the SM proceed via the left-handed charged current weak 
interaction, which results in a suppression of right-handed W bosons. The 
expected fractions of the three possible helicity states of the W boson are: lon- 
gitudinal fraction / s» 69%, left-handed fraction /_ ps 31% and right-handed 
fraction / + w 10~ 3 3\. The W boson helicity is reflected in the angular distri- 
bution cos 6** of its decay products, with 9* being the angle of the down-type 
decay products of the W boson (charged lepton, d- or s-quark) in the W boson 
rest frame relative to the top quark direction. Any observation of /+ at the 
percent level would signal new physics beyond the SM. 

DO measures the W boson helicity using a dataset corresponding to an inte- 
grated luminosity of 5.4 fb^ 1 , using both lepton + jets and dilepton final states 
of it decays g]. Since the measurement is still expected to be limited by sam- 
ple statistics, the event selection in each channel is optimised for it acceptance, 
using multivariate likelihood discriminants based on event topology and object 
kinematics. In the lepton + jets channel, the event kinematics is obtained from 
the best kinematic fit to the it event hypothesis, which enables the calculation of 
cos 9* for the leptonic W boson decay. The hadronic W boson decay is utilised 
as well to evaluate | cos 9* | , which does not permit discrimination between right- 
and left-handed W bosons, but still adds information to further constrain /o. 
In the dilepton events, m t — 172.5 GeV is assumed and cos9* is obtained for 
each top quark as the average over all possible solutions of the event kinematics, 
taking lepton and jet energy resolutions into account by varying the energies 
within their uncertainties. 

By comparing the observed angular distributions in data with templates of 
background and it signal with varying W boson helicity admixtures, /o and 
/+ can be extracted in a simultaneous fit (/_ = 1 — /+ — /o). This model- 
independent fit yields: 

fa = 0.669±0.078(stat.)±0.065(syst.), /+ = 0.023±0.041(stat.)±0.034(syst.), 
in excellent agreement with the SM expectation. 

Search for Neutral Current Top Decays 

Transitions between quarks of different flavour but identical charge (flavour 
changing neutral currents, FCNC) are forbidden at lowest order and heavily 
suppressed at higher orders in the SM. The decay t — » Zq (with q = u,c) thus 
is expected to occur with a branching fraction of 0(1O -14 ) [5], well out of reach 
of experimental sensitivity. Any observation of FCNC decays would indicate 
physics beyond the SM. 

DO performs a search for it — > WbZq + X decays based on a dataset corre- 
sponding to an integrated luminosity of 4.1 fb _1 , using the trilepton final state 
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I'vii + jets for the first time in this kind of search [BJ. This signature bene- 
fits from low background contributions, but also suffers from limited statistics. 
After a basic preselection reflecting the decay signature, signal and background 
(mainly diboson and V + jets production) are separated using distributions of 
jet multiplicity, scalar sum of the transverse momenta of leptons, jets and $t 
(Ht), and the reconstructed top quark mass from the t — > Zq decay. 

All observed distributions are consistent with the SM background contribu- 
tions, and consequently 95% C.L. limits on FCNC decays of it are derived: 

B(t ->■ Zq) < 3.2%, 

with an expected limit of 3.8 %. This is currently the world's best limit on 
B(t->Zq). 

Search for Fourth Generation t' Quark Pair Production 

While a fourth generation of fermions with a light neutrino (m v < mz/2) 
has been ruled out by electroweak precision data, the constraints on a fourth 
generation are much relaxed if the neutrino is sufficiently heavy. For example, a 
fourth generation t' quark mass of 400 GeV can be compatible with electroweak 
precision measurements, and the mass splitting relative to the corresponding 
b' quark is constrained to be small, leading to a preferred decay P — > Wq 

(g = d, «, 6) El- 
Based on a dataset with an integrated luminosity of 5.3 fb _1 , DO searches 
for pair production of fourth generation up-type quarks (PP), assuming the 
intrinsic P width is smaller than the detector resolution and the decay always 
proceeds via Wq [8]. Consequently, the t'P decay chain is identical to that 
of the top quark pair production, and the lepton + jets final state is used for 
the search with no 6-jet identification applied. To separate the signal from 
SM background processes, the observed events are compared with signal and 
background contributions in two-dimensional histograms of the scalar sum of 
lepton-p^, jet-px and $t (Ht) versus P mass rrif it from a kinematic fit to the 
event hypothesis. Based on the observed agreement between data and the SM 
contributions, 95% C.L. limits are derived on the PP production cross section 
as a function of my . 

While in the e + jets channel reasonable agreement between data and simu- 
lation is observed, in the \i + jets channel an excess of data over SM background 
is visible around m/u = 325 GeV. However, the data are best described with 
a t'P production cross section 3.2 ± 1.1 times the predicted cross section for 
rrif = 325 GeV. The probability of observing an excess of at least this size 
from the SM background processes alone corresponds to 2.5 standard devia- 
tions. Combining the observations from both channels, t'P production can be 
excluded at 95% C.L. for 

m? < 285 GeV, 
with an expected exclusion below 320 GeV. 
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Measurement of B{t -> Wb) /B(t ->■ Wq) 

The ratio of top quark branching fractions R can be expressed via CKM matrix 
elements as R = B(t->Wb)/ E q=diS>b B(t -> Wg)= |F t6 | 2 / E g=d , s , 6 |^ tg | 2 and 
provides therefore a measure of the relative size of \V t b\ compared to \V t d\ and 
|Vts|- Within the framework of the SM, assuming a unitary 3x3 CKM matrix 
and insignificance of non-VF boson decay modes of the top quark, the above 
expression simplifies to R = \Vtb\ 2 and R is strongly constrained from global 
CKM fits: R = 0.99830t$°o9 13- Deviations of R from unity could for example 
be a sign for the presence of a fourth heavy quark generation or non-SM top 
quark decay modes. 

DO measures R in a dataset corresponding to an integrated luminosity of 
5.4 fb _1 , using both the lepton + jets and dilepton it decay channels [TO] . 
In the lepton + jets channels, the number of identified 6-jets (0, 1, >2) is 
used together with a multivariate kinematic discriminant to separate the it 
signal from background in subsamples dominated by background in order to 
extract R simultaneously with the it production cross section <r t j, yielding R — 
0.95 ±0.07 (stat. + syst.) and a t t = 7.90t°;g7 (stat. + syst.) pb. In the dilepton 
channels, the lowest value of a neural network 6-jet identification algorithm 
applied to the leading jets in each event is used as a discriminant, yielding 
R = 0.86 ± 0.05 (stat. + syst.) and a ti = 8.19±o ^ (stat. + syst.) pb. These 
measurements allow the extraction of <j t i without assuming B(t — > Wb) = 1 as 
usually done in it cross section measurements. Combining both measurements 
yields: 

R = 0.90 ± 0.04 (stat. + syst.) and a ti = 7.74±°;^ (stat. + syst.) pb. 

The cross section measurement is in good agreement with the SM expectation, 
and the R measurement is within approximately 2.5 standard deviations of the 
SM expectation, being the most precise measurement to date. 

3 Summary 

DO is pursuing a wealth of top quark physics analyses that probe the validity of 
the SM, a few of which are presented in this report. So far, no evidence for new 
physics phenomena has been found in top quark decays. The search continues 
now both at the Tevatron and the LHC, and will include updates to the analyses 
presented here. 
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